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Abstract 

In this paper, I'll derive the Hamilton-Jacobi (HJ) equation for Mer- 
ton's problem in Utility Optimization Theory using a Calculus of Vari- 
ations (CoV) Approach. For stochastic control problems, Dynamic Pro- 
gramming (DP) has been used as a standard method. To the best of my 
knowledge, no one has used CoV for this problem. In addition, while 
the DP approach cannot guarantee that the optimum satisfies the HJ 
equation, the CoV approach does. 



1 Introduction 

Consider Merton's problem in Utility Optimization Theory. That is: 
given a stock with dynamics 

dSt = fiStdt + aStdWt 

(where Wt denotes the Brownian motion), find the amount Ht to invest 
in this stock that maximizes the expected utility at a terminal time T. 
Mathematically, if Xt represents the wealth, the problem to solve is 

maxE\U(XT\Xo], 

where U is the utility function {U is given and is concave). 

Suppose that the control function lit = 0!{t,Xt) depends only on 
the current state but not the pas10. Then the wealth has the following 
dynamic^: 

dXt = rXtdt + r)Utdt + aUtdWt 
= ait,Xt)dt + b{t,Xt)dWt, 



^Is it a reasonable assumption if the stock price follows a Brownian motion, i.e. it has 
Markov property? 

^For simplicity, let's assume that fi and a are independent of the stock price. 
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where 



a{t, Xt) = rXt + (/X - r)a(t, Xt) (2) 
b{t,Xt) = aa{t,Xt). (3) 

The Fokker-Planck (FP) equation, which describes the evolution of 
probability density p{t,x) of the random process X^, has the form 

dp d{ap) ld'^{b'^p) 

dt " ^ 2 9x2 ' ( ) 

with the initial condition p{0,x) = po{x) = 5{x — Xq). 

Our goal is to maximize E[C/(Xt)|Xo] = J^U{x)p{T, x)dx over all 
admissible control functions a{t,Xt). Since the utility expectation is a 
concavjl functional of the control, the optimum can be determined by 
the CoV method. This is actually a PDE-constrainejl optimization. 



2 Calculus of Variations 



Let's introduce the Lagrange multipliers X{t,x) and Ao(x) and consider 
the problem of optimizing 

L{p,a,X,Xo) = / U{x)p{T,x)dx + / Xo{x)[p{0, x) - pQ{x)]dx 



+ 



d d 



1 



dt dx 



2dx^ 



dxdt. 



1. Taking the first variation of L with respect to p, we obtain 



5pL 



U{x)6p{T,x)dx + I XQ{x)6p{0,x))dx 

29x2 
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—Sp+ — 
dt dx 



dxdt. 



Note, by integration by parts, that 



d f \T 

X—dpdxdt =( / X6pdx)\„ - 
JR ot Jo 



XtSpdxdt 



X{T,x)6p{T,x)dx — I X{0,x)6p{0,x)dx 

T 

XtSpdxdt, 







^Underlined statement can be proved but is not shown in this paper, 
""■the density function is constrained by the FP equation 
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/ \-^(adp)dxdt = [ Xadp\^°°dt— f f XxaSpdxdt 
Jo dx Jo Jo Jr 



■- / \{t,+oo)a{t,+oo)5p{t,+oo)dt 
Jo 

— / \{t, —oo)a{t, —oo)6p{t, —oo)dt 
Jo 

T f 

XxaSpdxdt, 



II 

Jo Jm 



and 



+ / / ^b'Spdxdt. 

./TCP ^ 



If 6pL = for all admissible 6p then 

X{T,x) =-U{x) 
Xo{x) =X{0,x) 

Xt + aXx + -^Xxx —0, 

and some boundary (at infinity) conditions must be satisfied. 

2. Taking the first variation of L with respect to a (which occurs only 
in a and 6), we obtain 



f f d I f d 

SaL = / X—[{fi — r)pSa]dxdt — / / X-^-^{a^paSa)dxdt 
Jo Jr Jo Jr ox 

= / X{fi — r)p6a\^°° dt — / / Xxifi — r)pdadxdt 
Jo °° Jo Jm. 

— / X—(a'^paSa)\^°°+ / / Xx-:^(.cr^pc(6a)dxdt 
Jo ox Jo Jr dx 

= / X{iJ, — r)p5a\'^°° dt — / / Xxin — r)p5adxdt 
Jo °° Jo Jr 

+ / Xx(T^p6ce\^°°dt — / / XxxO'^poiSadxdt. 
Jo °° Jo Jr 

If SaL = for all admissible a then 

Xxxcr^pa + XxilJ:- r)p = 0, 

or (T^AxxCt + (/i — r)Xx = ahnost everywhere for p ^ a.e.) and 
some boundary conditions must be satisfied. 

3. Setting 5\L = and 5\qL = gives us the Fokker-Planck equation 
with the initial condition of p. 
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Since L is concave , the necessary and sufficient condition for optimality 
can be described by the following equations (with boundary conditions 
at infinity not stated here). 

• State equations: 

dp d{ap) 1 d'^ib'^p) 
p(0,x) =(5(x-Xo). (6) 

• Adjoint equations: 

X{T,x) = -U{x), (7) 
62 

Ai + aXx + -Xxx =0. (8) 

• Control equation: 

a'^Xxxd + (a* - r)Xx = 0. (9) 
The control equation implies that 



a = (10) 



and hence 



a{t, x) =rx 
b{t,x) = - 



(^-r)2A, 



X 



CT^X 

if^-r )X 
aXx 



XX 
X 



Plugging these into the adjoint equation, we obtain a Hamilton-Jacobi 
(HJ) equation of A: 

Xt + rxXx-^A^^P^ = 0. (11) 
/a Axx 

The Lagrange multiplier A is essentially the same as the function value 
V in the Dynamic Programming (DP) approactH. 



3 Discussions 

• For variable drift and volatility (i.e. they depend on the stock 
price), the approach can be extended easily by using the two di- 
mensional version of FP equation (actual derivation will be shown 
later). For stochastic volatility? I'm not sure but will think more 
about it. 

^In DP approach, the value function V{t, x) denotes the optimized expected utility at T 
given initial wealth x at time t. 
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The method can also be extended easily if we want to optimize the 
expected utility over a period of time instead of at the terminal 
time. But it's likely that the result can also be derived easily by 
means of DP. 



The CoV approach 



In DP approach, if we can solve the derived HJ equation then we 
know that it's the optimum. However, it doesn't guarantee that 
the optimum has to satisfy the HJ equati 
shows that A has to satisfy the HJ equati 

How do we explain that the Lagrange multiplier A is the value func- 
tion? This fact seems to have been well explained by economists. 
(Baxley and Moohouse, 'Lagrange multiplier problems in economies'. 
The Am. Math. Monthly.) 



4 Numerical methods for computing vis- 
cosity solution 

This is what I currently want to study. 



5 Derivation of the Fokker-Planck equa- 
tion 

In this section, we'll derive the one-dimensional Fokker-Planck equation 
that describes the evolution of density of the stochastic variable Xt with 
dynamics 

dXt = fi{t, Xt)dt + a{t, Xt)dWt. 

For any function f{x), consider {f{Xt)) = ■^'E[f{Xt)] at an arbitrary 
time t. On one hand, 

-^(/(Xi) = ^ / pit,x)fix)dx= [ pt{t,x)f{x)dx. 

On the other hand, 



--{f,{t,Xt)f, + -a\t,Xt)f, 



p{t, X 
fix) 



XX I 

1 



(by Ito's formula) 

dx 



H{t,x)f^{x) + ^cr'^it,x)fxxix) 
^ . ^ 1 2 ^ ^ 



(integration by parts). 



^If this sounds mysterious, please refer "Robert C. Merton, 1973. An Intertemporal 
Capital Asset Pricing Model. Econometrica 41-' 867-887". 

^It has been shown Crandall and Lions that for HJ-type equations, the viscosity solution, 
which is the appropriate solution in this context, exists uniquely. 
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Since the two formulations hold for any function /, 
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